Section: Partnerships and Cooperations

European Initiatives

FP7 & H2020 Projects

ERC-CLIM

Participants : Pierre Allain, Pierre David, Elian Dib, Simon Evain, Christian Galea, Christine Guillemot, Laurent Guillo, Fatma Hawary, Xiaoran Jiang, Maja Krivokuca, Ehsan Miandji, Hoai Nam Nguyen, Mira Rizkallah, Alexander Sagel, Jinglei Shi.

• Title : Computational Light field Imaging.

• Research axis : 7.1.1, 7.1.2, 7.1.4, 7.2.1, 7.2.3, 7.2.4, 7.2.2, 7.3.1, 7.3.2, 7.3.3, 7.3.4

• Partners : Inria-Rennes

• Funding : European Research Council (ERC) advanced grant

• Period : Sept. 2016 - Aug. 2021.

All imaging systems, when capturing a view, record different combinations of light rays emitted by the environment. In a conventional camera, each sensor element sums all the light rays emitted by one point over the lens aperture. Light field cameras instead measure the light along each ray reaching the camera sensors and not only the sum of rays striking each point in the image. In one single exposure, they capture the geometric distribution of light passing through the lens. This process can be seen as sampling the plenoptic function that describes the intensity of the light rays interacting with the scene and received by an observer at every point in space, along any direction of gaze, for all times and every wavelength.

The recorded flow of rays (the light field) is in the form of high-dimensional data (4D or 5D for static and dynamic light fields). The 4D/5D light field yields a very rich description of the scene enabling advanced creation of novel images from a single capture, e.g. for computational photography by simulating a capture with a different focus and a different depth of field, by simulating lenses with different apertures, by creating images with different artistic intents. It also enables advanced scene analysis with depth and scene flow estimation and 3D modeling. The goal of the ERC-CLIM project is to develop algorithms for the entire static and video light fields processing chain. The planned research includes the development of:

• novel low-rank or graph-based models for dimensionality reduction and compression

• deep learning methods for scene analysis (e.g. scene depth and scene flow estimation)

• learning methods for solving a range of inverse problems: denoising, super-resolution, axial super-resolution, view synthesis.